1. Field of the Invention
This invention relates to signal processing and, more particularly, to novel systems and methods for pattern recognition and data interpretation.
2. The Background Art
Environmental data and stimuli have been the subject of much study for the purpose of organizing, interpreting and making useful for a future application the information that can be learned from the data.
Some sources of data that have captured the interest of those skilled in the art include neural functions. A neurocognitive adaptive computer interface method and system based on on-line measurement of the user""s mental effort is described in U.S. Pat. No. 5,447,166. This method appears to be a neural network algorithm trained on data from a group of subjects performing a battery of tasks to estimate neurocognitive workload. It seems to represent a very specific algorithm trained on group data to estimate another very specific cognitive feature of brainwaves. It does not appear to be a general purpose method of analyzing all brain activity, nor does it appear to have broad application outside the tasks on which it was trained.
An electroencephalic neurofeedback apparatus for training and tracking of cognitive states is described in U.S. Pat. No. 5,406,957. This patent describes the basic invention of the mind mirror which is commercially available. The brainwave signal is Fast-Fourier-Transformed and the resulting frequency bands are displayed on a computer. The display and signal are used for biofeedback purposes but the signal is not classified or interpreted.
A brainwave directed amusement device was developed as described in U.S. Pat. No. 5,213,338 in a patent that details an arousal-level detection algorithm which is used to provide simple control of a video game. The algorithm measures the intensity (by amplitude) of raw brainwaves or a particular frequency band which varies in amplitude with the degree of arousal or relaxation a player experiences. This device is similar to other products on the market that use arousal-level to control a video game. The algorithm which provides this type of control appears limited to emotion-based arousal-level estimation and is correspondingly capable of only simple control through changes in the amplitude of one or two frequency bands from one or two sensor channels. These types of algorithms are typically limited to providing control based on brainwave arousal level or Galvanic Skin Response (GSR), also known as skin conductivity. This type of algorithm may provide a functional polygraph for lie detection and emotional arousal-level monitoring.
U.S. Pat. No. 5,392,210 concerns the localization or estimated reconstruction of current distributions given surface magnetic field and electric potential measurements for the purpose of locating the position of electrophysiological activities. The patent describes a method of getting closer to the source activity, but does not seem to provide a system of analysis or classification or interpretation of that source activity.
A method and device for interpreting concepts and conceptual thought from brainwave data and for assisting diagnosis of brainwave dysfunction is described in U.S. Pat. No. 5,392,788. This patent describes an analysis of Average Evoked Potentials by comparing the measured Evoked Potentials to the size and shapes of model waveforms or Normative Evoked Potentials, yielding from the comparison an interpretation. However, the system averages data, and requires an a priori model to be constructed for a diagnosis to be possible.
In view of the foregoing, it is a primary object of the present invention to provide a novel apparatus and methods for signal processing, pattern recognition, and data interpretation.
It is also an object of the present invention to find attributes of a signal that may be correlated with an event associated with the same time segment as the signal where correlations are found by manipulating the signal data with various operators and weights to xe2x80x9cexpand the signalxe2x80x9d into many different features.
Further, it is an object of the present invention to process each signal piece or segment occurring over a time segment to determine correlations between a known event and a particular, processed xe2x80x9cfeature segment.xe2x80x9d
It is still a further object of the present invention to determine optimal ways to manipulate a signal for purposes of distinguishing an event from the signal.
In addition, it is an object of the present invention to learn from at least two patterns or two event types derived from data collected from a series of related chronological events.
Another object of the present invention is to analyze complex data, from whatever source (see below for exemplary sources), and classify and interpret the data.
Consistent with the foregoing objects, and in accordance with the invention as embodied and broadly described herein, an apparatus called a signal interpretation engine is disclosed in one embodiment of the present invention as including a computer programmed to run a plurality of modules comprising a feature expansion module, a consolidation module, and a map generation module.
The feature expansion module contains feature operators for operating on a signal to expand the signal to form a feature map of feature segments. Each feature segment corresponds to a unique representation of the signal created by a feature operator operating on the signal across an epoch. An epoch corresponds to a time segment or to an event occurring within a time segment. The invention further comprises a weight table module that provides a weight table having weight elements. Each weight element has a weight corresponding to a feature segment of the feature map.
The consolidation module provides a superposition segment that combines the feature segments of the feature map corresponding to the epoch by forming an inner product of the feature map and the weight table. The consolidation module also applies aggregators to consolidate the inner products into a distribution function representing an attribute over a domain reflecting a selected weight table, aggregator, and event type, corresponding to each value of the attribute. The map generation module produces an interpretation map that reflects a preferred weight table and aggregator to be applied to the signal data to characterize the event.
A method for providing an interpretation map may include the steps of providing signal data corresponding to an event; expanding the signal data by applying a feature operator to create feature segments; providing a weight table comprising weight elements. Each weight element having a weight for adjusting the relative influence of each of the feature segments with respect to one another; superimposing one or more feature segments to provide a superposition segment by means of forming an inner product of feature segments and weight elements; aggregating the superposition segment to a attribute value; organizing attribute values from many epochs to provide a distribution function relating a value to an event type, an event instance, a weight table, an aggregator operator; and generating an interpretation map reflecting parameters for optimizing the feature expansion, consolidation, and classification of signal data into event types.
The signal data for the apparatus or the method can be derived from a medical context, a research context, and an industrial context. The medical context can be a disease, a physical impairment, a mental impairment, a medical procedure, or a therapy or treatment. The disease can be cancer, autoimmunity, a disease related to a cardiovascular condition, a viral infection, a neurological disease, a degenerative disease, a disease correlated with aging, or a disease correlated with stress. The research context can be single-trial analysis, cognitive study, psychology, neurology, cardiology, oncology, study of sleep, study of breathing, study of body conductance, study of body temperature, plant study, insect study, animal study, pharmaceutical drug-effect study, population study, flow study, physical environment study, geology, seismology, astronomy, medical clinical research, molecular biology, neuroscience, chemistry, or physics. The industrial context can be individual identification for security purposes, drug evaluation and testing, lie detection, vehicle vibration analysis, temperature diagnostics, fluid diagnostics, mechanical system diagnostics, industrial plant diagnostics, radio communications, microwave technology, turbulent flow diagnostics, sonar imaging, radar imaging, audio mechanism, yield optimization, efficiency optimization, natural resource exploration, information exchange optimization, traffic monitoring, spatial interpretation, or toxicology.
The interpretation map generated by the apparatus or the task performed by the method can provide a control mechanism based on an event or series of events selected from the group consisting of rehabilitation, biofeedback, real-time and hands-free control of virtual and real objects, mind mouse and thinking cap for controlling objects, neural controlled devices and video games, muscle controlled devices and video games, conductivity controlled video games using skin conductance, hands-free voice-free computer assisted telepathy, communication for the deaf, mute, blind or severely disabled, or a mechanism aiding prosthetic use and control.
The interpretation map generated by the apparatus or the task performed by the method can generate a prediction based on an event or series of events comprising an area of observation and monitoring selected from the group consisting of meteorology, a stock market, geology, astronomy, seismology, genetics, neurology, cardiology, or oncology. The apparatus further includes a computer display, and the method further provides using a computer display. Additionally, the invention provides a method of labeling signal data by event type.
Further, the invention is a method of creating useful applications of the signal interpretation engine, the method comprising measuring and recording events and event types, measuring and recording signal data, establishing a correspondence between an event and a signal data epoch, labeling signal data epochs by event type, using labeled signal data epochs in a learning system to generate an interpretation map, using signal data and the interpretation map in a classification system to produce interpretations, and using the interpretations to provide a useful result.